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DETAILED ACTION 



Response to Amendment 

1 . Examiner has acknowledged amendments submitted January 1 7, 2008. 

a. Claims 1 and 3-8 have been amended. 

b. Claim 2 has been cancelled. 



Response to Arguments 

2. Applicant's arguments with respect to claims 1 , and 3-12 have been considered 
but are moot in view of the new ground(s) of rejection. 



Double Patenting 

3. Examiner acknowledges applicant's request to address any provisional 
obviousness-type double patenting issues once the rejection of the claims 1-12 are 
resolved. 

The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. A nonstatutory 
obviousness-type double patenting rejection is appropriate where the conflicting claims 
are not identical, but at least one examined application claim is not patentably distinct 
from the reference claim(s) because the examined application claim is either anticipated 
by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 
F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 
USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 
1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 
F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 
644 (CCPA 1969). 
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A timely filed terminal disclaimer in compliance with 37 CFR 1 .321 (c) or 1 .321 (d) 
may be used to overcome an actual or provisional rejection based on a nonstatutory 
double patenting ground provided the conflicting application or patent either is shown to 
be commonly owned with this application, or claims an invention made as a result of 
activities undertaken within the scope of a joint research agreement. 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 

4. Claim 1 and 9 are provisionally rejected on the ground of nonstatutory 
obviousness-type double patenting as being unpatentable over claim 15 and 18 
respectively of copending Application No. 10/685,701 in view of Nielsen (U.S. Patent 
No. 5,897,644). 

Regarding claim 1 : An apparatus used to process audio and/or video data 
using a markup document which comprises: a) decoding AV data; and b) 
processing a markup document which includes the steps of 1) obtaining device 
aspect ratio information, 2) obtaining (transforming) the markup picture according 
to the device aspect ratio, 4) combining the markup picture and AV picture; 5) 
displaying (outputting) the interactive picture which includes the AV picture, is an 
obvious pre-requisite for processing operating instructions encoded on a 
computer readable medium (see [col. 4, II. 20-29] for a program product 
embedded on a computer usable medium). 

Regarding claim 9: An apparatus used to process a markup document in 
an interactive mode which comprises: a) controlling a device to playback (output) 
video picture from AV data; b) controlling a device to transform a markup 
document according to device-aspect-ratio information on an aspect ratio of a 
target display device in the interactive mode is an obvious pre-requisite to 
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implement the method in a computer system used to process audio video data in 
an interactive mode (see [col. 3, II. 29-50] for a method implemented by an 
apparatus). 

5. Claim 1 is provisionally rejected on the ground of nonstatutory obviousness-type 
double patenting as being unpatentable over claim 1 of copending Application No. 
10/536,911 in view of Nielsen (U.S. Patent No. 5,897,644). 

Regarding claim 1: Claim 1 of copending Application No. 10/536,911 
teaches an apparatus for reproducing (outputting) an interactive content an 
interactive markup document which includes AV data. Information from the 
markup document is received (determinations are made according to aspect ratio 
information and resolution information from the markup document). Claim 1 also 
teaches a pre-process of enlarging or reducing a size of the font according to the 
determined pixel aspect ratio. Although claim 1 does not explicitly teach 
"transforming" it would be obvious that one is essentially transforming by 
enlarging or reducing the size of a font and it is also obvious that the markup 
document as a whole could be "transformed" in the same manner as the fonts 
within the markup document (see [figs. 9a-9c], [col. 9, II. 6-44], [col. 10-11, II. 57- 
4]). 

6. Claim 1 is provisionally rejected on the ground of nonstatutory obviousness-type 
double patenting as being unpatentable over claim 20 of copending Application No. 
10/280,127 in view of Nielsen (U.S. Patent No. 5,897,644). 
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Regarding claim 1: Claim 20 of copending Application No. 10/280,127 teaches a 
method of reproducing (displaying) screen display files which realize interactive 
environments (a markup document) so as to be adjusted (transformed) to various 
screen sizes comprising: reading a display screen aspect ratio set in a reproducer; and 
reading a screen display file having the display screen aspect ratio from the storage 
medium and displaying an interactive screen based on the read screen display file. 
Claim 1 of the instant application is obvious over claim 20 because claim 1 recites 
obtaining device-aspect-ratio information (reading a display screen aspect ratio), 
obtaining and displaying a markup picture and AV picture from data in the display file 
(displaying an interactive screen based on the read screen display file). 

This is a provisional obviousness-type double patenting rejection because the 
conflicting claims have not in fact been patented. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1 , 3-9, 11 and 1 2 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Nishi (U.S. Patent No. 6,681 ,395 B1 ) in view of Nielsen (U.S. Patent 
No. 5,897,644). 
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Regarding amended claim 1: Nishi discloses an apparatus which 
processes audio and/or video (AV) data in an interactive mode using a markup 
document, comprising: an AV playback engine which decodes the AV data to 
output an AV picture; and an enhanced audio and/or video (ENAV) engine which 
interprets the markup document to read device-aspect-ratio information included 
in the markup document and to obtain a source markup picture (see [fig. 1 items 
42-44], [col. 4, II. 17-26], [col. 4, II. 51-63], [col. 6, II. 1-24], [col. 8, II. 31-33] for a 
playback engine which decodes AV data and the markup document which 
contains device-aspect-ratio information to output to an AV picture), combines 
the markup picture and the AV picture, and outputs an interactive picture 
including the markup picture and the AV picture (see [fig. 1], [fig. 3,4,6], [fig. 12], 
[col. 6, II. 10-24] for using a template containing device-aspect-ratio information 
to output the markup template (picture) and the AV picture). 

Nishi does not explicitly teach "transforming" the source markup picture. 

Nielsen teaches transforming a markup picture to fit within a display view, 
with a layout similar to the original, that is of a size other than that of the display 
view used to construct the original image (see [abstract], [col. 3, II. 29-50]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to transform a source markup picture, as taught in Nielsen, 
according to the aspect ratio and the size of the display screen used to display 
the picture, as taught in Nishi, because using a transformation process allows the 
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presentation of fixed canvas presentations on display devices of differing sizes 
(see [col. 1, II. 9-15], [col. 3, 29-37]). 

Regarding amended claim 3: Nishi discloses the apparatus of claim 1, 
wherein the device-aspect-ratio information is information on an aspect ratio of a 
screen of a target display device intended to display the markup document in the 
interactive mode (see [col. 1, II.63-67], [col. 2, II. 57-63], [col. 6, II. 10-35] where 
the aspect ratio is according to the display screen of the user's television set). 

Regarding amended claim 4: Nishi discloses the apparatus of claim 1 , 
wherein the ENAV engine obtains the device-aspect-ratio information which is 
information on an aspect ratio of a screen of a target display device intended to 
display the markup document in the interactive mode (see the rejection of claim 
3). 

Nishi does not teach wherein the markup document is displayed according 
to a design of a markup document maker. 

Nielsen teaches wherein the markup document is displayed such that a 
transformation used to either magnify or reduce the canvas preserves the aspect 
ratio of the fixed canvas such that the page layout is similar to the original (see 
[abstract], [col. 3, 29-67] for reading on displaying the canvas according to a 
design of a markup document maker). 

At the time the invention was made, it would have been obvious to 
one of ordinary skill in the art to display the canvas (markup document) according 
to its original design (whereby the original design was derived by a designer), as 
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taught in Nielsen, when using methods to display a markup document in the most 
suitable form, as taught in Nishi, because this process allows the presentation of 
fixed canvas presentations to fit on display devices of differing sizes (see [col. 1, 
II. 9-15], [col. 3, 29-37]). 

Regarding amended claim 5: Nishi discloses the apparatus of claim 1, 
wherein the ENAV engine parses the device-aspect-ratio information which is 
written in the markup document using a property of a tag (see [fig. 12], [col. 2, II. 
57-63], [col. 4, II. 17-20 and II. 51-62], [col. 6, II. 1-35], [col. 9], [col. 11, II. 13-20]). 

Regarding amended claim 6: Nishi does not explicitly teach wherein the 
ENAV engine "transforms" the source markup picture into the markup picture 
with a predetermined aspect ratio according to the device-aspect-ratio 
information of the markup document. 

Nielsen teaches transforming a markup picture with a predetermined 
aspect ratio to fit within a display view, with a layout similar to the original, that is 
of a size other than that of the display view used to construct the original image 
(see [abstract], [col. 3, II. 29-50]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to transform a source markup picture which contains a 
predetermined aspect ratio, as taught in Nielsen, according to the aspect ratio 
and the size of the display screen used to display the picture, as taught in Nishi, 
because using a transformation process allows the presentation of fixed canvas 
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presentations on display devices of differing sizes in a manner similar to the 
original image (see [abstract], [col. 1, II. 9-15], [col. 3, 29-37]). 

Regarding amended claim 7: Nishi discloses the apparatus of claim 1, 
wherein the ENAV engine displays the source markup picture as a markup 
picture with an aspect ratio of 4:3 or 16:9 according to the device-aspect-ratio 
information of the markup document (see [col. 6, II. 10-24]). 

Nishi does not explicitly teach "transforming" the source markup picture. 

Nielsen teaches transforming a markup picture to fit within a display view, 
with a layout similar to the original, that is of a size other than that of the display 
view used to construct the original image (see [abstract], [col. 3, II. 29-50]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to transform a source markup picture, as taught in Nielsen, 
according to the aspect ratio and the size of the display screen used to display 
the picture, as taught in Nishi, because using a transformation process allows the 
presentation of fixed canvas presentations on display devices of differing sizes 
(see [col. 1, II. 9-15], [col. 3, 29-37]). 

Regarding amended claim 8: Nishi does not disclose wherein the ENAV 
engine scales the source markup picture to output the markup picture 
corresponding to the device- aspect-ratio information of the markup document in 
response to information on an aspect ratio of a destination device being different 
from the device-aspect-ratio information, the destination device substantially 
displaying the markup document and the device-aspect-ratio information being 
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data including information on an aspect ratio of a target display device intended 
to display the markup document in the interactive mode. 

Nielsen teaches scaling a markup picture using a transformation 
mechanism which maps the HTML data stream to fit within a display based on 
the height and width dimensions and aspect ratio of the html canvas, and 
comparing the calculated values to the size of the area on the screen accessible 
for view such that a layout similar to the original, that is of a size other than that 
of the display view used to construct the original image can be displayed (see 
[abstract], [col. 3, II. 29-67], [col. 8, II. 20-65]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to scale a source markup picture according to the aspect 
ratio predefined in the markup document using a transformation value "T", as 
taught in Nielsen, according to the aspect ratio and the size of the display screen 
used to display the picture, as taught in Nishi, because using a transformation 
process to scale a markup document allows the presentation of fixed canvas 
presentations on display devices of differing sizes in a form similar to the original 
markup document (see [col. 1 , II. 9-15], [col. 3, 29-37]). 

Regarding claim 9: Nishi discloses an apparatus for processing a markup 
document in an interactive mode, comprising: a controller to output a markup 
picture of the markup document and a video picture of an audio and/or video 
data in the interactive mode [see [fig. 1 items 42 and 45]). 
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Nishi does not explicitly teach a markup "transformer" which transforms 
the markup picture according to device-aspect- ratio information corresponding to 
the markup document, the device-aspect-ratio information being data including 
information on an aspect ratio of a target display device intended to display the 
markup document. 

Nielsen teaches a transformer used for transforming a markup picture 
using a transformation mechanism which maps the HTML data stream to fit 
within a display based on the height and width dimensions and aspect ratio of the 
html canvas, and comparing the calculated values to the size of the area on the 
screen accessible for view such that a layout similar to the original, that is of a 
size other than that of the display view used to construct the original image can 
be displayed (see [fig. 2], [abstract], [col. 3, II. 29-67], [col. 8, II. 20-65]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to transform a source markup picture according to the 
aspect ratio predefined in the markup document using a "transformer", as taught 
in Nielsen, according to the aspect ratio and the size of the display screen used 
to display the picture, as taught in Nishi, because using a transformation process 
to scale a markup document allows the presentation of fixed canvas 
presentations on display devices of differing sizes in a form similar to the original 
markup document (see [col. 1 , II. 9-15], [col. 3, 29-37]). 

Regarding claim 11: Nishi discloses the apparatus of claim 9, wherein 
the device-aspect-ratio information is included in the markup document (see [fig. 
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12], [col. 6, II. 10-35], [col. 8, II. 31-33] where the device aspect ratio information 
is included by representative values "N" or "W"). 

Regarding claim 12: Nishi does not teach wherein the markup 
transformer scales the markup picture according to the device-aspect-ratio 
information in response to information. 

Nielsen teaches scaling a markup picture using a transformation 
mechanism which maps the HTML data stream to fit within a display based on 
the height and width dimensions and aspect ratio of the html canvas, and 
comparing the calculated values to the size of the area on the screen accessible 
for view such that a layout similar to the original, that is of a size other than that 
of the display view used to construct the original image can be displayed (see 
[abstract], [col. 3, II. 29-67], [col. 8, II. 20-65]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to scale a source markup picture according to the aspect 
ratio predefined in the markup document using a transformation value "T", as 
taught in Nielsen, according to the aspect ratio and the size of the display screen 
used to display the picture, as taught in Nishi, because using a transformation 
process to scale a markup document allows the presentation of fixed canvas 
presentations on display devices of differing sizes in a form similar to the original 
markup document (see [col. 1 , II. 9-15], [col. 3, 29-37]). 
8. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Nishi in 
view of Nielsen and Rainville et al. (U.S. Pre- Grant Pub. No. 2002/006941 1 A1). 
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Regarding claim 10: Nishi does not disclose the apparatus of claim 9, 
wherein the controller embeds the video picture in the markup picture according 
to embedding information of the markup document. 

Rainville teaches embedding a video picture in a markup document 
according to embedding information of the markup picture (see [figs. 2A-4B], [fig. 
6], [fig. 7], [fig. 8], [0009], [0043-55]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art that it is possible to embed a video into a markup 
document, as taught in Rainville, before changing the markup document to fit the 
display device, as taught in Nishi, because adding embedded video enhances 
markup documents (web pages) (see [0008]). 

Conclusion 

9. Because new grounds of rejections have been made to reject claims 1 and 3-12, 
this Office Action is made Non-Final. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jason Thomas whose telephone number is (571) 270- 
5080. The examiner can normally be reached on Mon. - Thurs., 8:00 a.m. - 5:00 p.m.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Koenig can be reached on (571) 272-7296. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

J. Thomas 
/Andrew Y Koenig/ 

Supervisory Patent Examiner, Art Unit 2623 



